I’ve talked about the TechEmpower performance benchmarks before, but I wanted to highlight them again. They are the closest thing the internet has to a giant cage match between nearly every language and framework out there. The benchmarks have a suite of tests, such as run 20 queries on a page and output some data, and every language and framework implements the same logic in their syntax and style. The tests literally take days to run in full and spin up each combination of language and framework in docker containers where they are hammered with oodles of traffic and then the juicy stats are recorded for sweet graphical comparisons.
I added the following to the site a year or so ago:
- Raw Lucee server
- Raw Adobe ColdFusion server
- ColdBox MVC running on Lucee
- ColdBox MVC running on Adobe ColdFusion
The site is basically information overload. There’s just dozens and dozens of combinations of languages, frameworks, databases, web servers, etc-- and many of them are crazy fast micro frameworks you’ve never heard of which are pretty cool. You can apply a huge list of filters to try and carve down the list of frameworks to a useful size of equivalent ones.
The Techempower benchmarks run a huge “official” benchmark every year or so, but have daily builds that basically run 24/7. They haven’t done an official build since I added CFML, but you can see how it performs in the daily runs.
Let me be the first to say my filters are pretty arbitrary. CFML does better on more complex pages with more queries than other languages. That’s because it’s got a little more overhead for a simple Hello World request (we’re talking ms here) but it’s JVM concurrency and datasource connection pooling really shine on a more complex test. As such, the link and screenshot above is for the “Data Updates” test, which is described here:
This test exercises database writes. Each request is processed by fetching multiple rows from a simple database table, converting the rows to in-memory objects, modifying one attribute of each object in memory, updating each associated row in the database individually, and then serializing the list of objects as a JSON response. The test is run multiple times: testing 1, 5, 10, 15, and 20 updates per request. Note that the number of statements per request is twice the number of updates since each update is paired with one query to fetch the object. All tests are run at 512 concurrency.
I also rather arbitrarily chose a selection of languages and frameworks that I felt were in competition with CFML. Feel free to play with the filters yourself. Let’s discuss the results of this test.
- Go is very fast. This is no surprise as Go is designed to be as small as possible and even discourages use of frameworks all together. I couldn’t get the filter to only show one of the Go configurations, but you can see it’s the only language that was as fast or faster than CFML in this test!
- CFML basically came in second place out of the selected languages and frameworks. Raw CFML is faster than ColdBox as expected but it’s not a massive difference.
- Node.js came in slower than both raw CFML and ColdBox MVC
- Groovy (Grails) came in slower than both raw CFML and ColdBox MVC
- Ktor jasync (Kotlin) came in slower than both raw CFML and ColdBox MVC
- Ruby on Rails came in slower than both raw CFML and ColdBox MVC
- Laravel (PHP) came in slower than ColdBox MVC which it is equivalent to. There’s a million PHP frameworks, I picked this one because I know it’s very popular and modern.
- Django (Python) came in dead last by a long shot
Not all the test results are the same. Play around with the site to compare your favorite languages and see how they hold up in the simple hello world tests vs the heavy lifting DB tests. I’ve stacked the cards a bit in my selections above, but I think it’s more indicative of a real world web app if we’re honest.
Let me just say, how freaking awesome is it to see CFML (both Lucee and Adobe) blowing the pants off other popular web frameworks. I think this sort of head-to-head comparison is great information to use when defending CFML as a battle-tested production server.
Note, if you’d like to see the code and Docker setup, feel free to poke around the repo.
